InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective

Continual learning (CL) aims to constantly learn new knowledge over time while avoiding catastrophic forgetting on old tasks. We focus on continual text classification under the class-incremental setting. Recent CL studies have identified the severe performance decrease on analogous classes as a key factor for catastrophic forgetting. In this paper, through an in-depth exploration of the representation learning process in CL, we discover that the compression effect of the information bottleneck leads to confusion on analogous classes. To enable the model learn more sufficient representations, we propose a novel replay-based continual text classification method, InfoCL. Our approach utilizes fast-slow and current-past contrastive learning to perform mutual information maximization and better recover the previously learned representations. In addition, InfoCL incorporates an adversarial memory augmentation strategy to alleviate the overfitting problem of replay. Experimental results demonstrate that InfoCL effectively mitigates forgetting and achieves state-of-the-art performance on three text classification tasks. The code is publicly available at https://github.com/Yifan-Song793/InfoCL.Comment: Findings of EMNLP 2023. An improved version of arXiv:2305.0728

Relating the composition of continental margin surface sediments from the Ross Sea to the Amundsen Sea, West Antarctica, to modern environmental conditions

Investigating the multiple proxies involving productivity, organic geochemistry, and trace element (TE) enrichment in surface sediments could be used as paleoenvironment archives to gain insights into past and future environmental conditions changes. We present redox-sensitive TEs (Mn, Ni, Cu, U, P, Mo, Co, V, Zn, and Cd), productivity-related proxies (total organic carbon and opal), and total nitrogen and CaCO3 contents of bulk surface sediments of this area. The productivity proxies from the shelf and coastal regions of the Ross and the Amundsen seas showed that higher productivity was affiliated with an area of nutrient-rich deep water upwelling. The upwelling of weakly corrosive deep water may be beneficial for preserving CaCO3, while highly corrosive dense water, if it forms on the shelf near the coastal region (coastal polynya), could limit the preservation of CaCO3 in modern conditions. There were no oxic or anoxic conditions in the study area, as indicated by the enrichment factors of redox-sensitive TEs (Mn, Co, and U). The enrichment factor of Cd, which is redox-sensitive, indicated suboxic redox conditions in sediment environments because of high primary productivity and organic matter preservation/decomposition. The enrichment factors of other redox-sensitive TEs (P, Ni, Cu, V, and Zn) and the correlations between the element/Ti ratio with productivity and nutrient proxies indicated that the organic matter decomposed, and there was massive burial of phytoplankton biomass. There was variation in the enrichment, such that sediments were enriched in P, Mo, and Zn, but depleted in Ni, Cu, and V

Autistic clinical profiles, age at first concern, and diagnosis among children with autism spectrum disorder

BackgroundTo explore the relationship between autistic clinical profiles and age at first concern and diagnosis among children with autism spectrum disorder. The clinical profiles included the severity of autism, cognition, adaptability, language development, and regression.MethodsThe multivariate linear regression model was used to examine the association of diagnostic age and first-concern age with autistic clinical profiles and with further stratification analysis.ResultsA total of 801 autistic children were included. Language delay and regression were associated with earlier diagnostic age (language delay: crudeβ: −0.80, 95%CI%: −0.92–−0.68; regression: crudeβ: −0.21, 95%CI%: −0.43–−0.00) and the age of first concern of autistic children (language delay: crudeβ: −0.55, 95%CI%: −0.65–−0.45; regression: crudeβ: −0.17, 95%CI%: −0.34–−0.00). After stratification by sex, language delay tended to be more associated with the earlier diagnostic age among boys (crudeβ: −0.85, 95%CI%: −0.98–−0.72) than among girls (crudeβ: −0.46, 95%CI%: −0.77–−0.16). After stratification by maternal education level or family income level, language delay was most associated with the earlier diagnostic age in autistic children from families with higher socioeconomic levels.ConclusionLanguage delay, rather than other symptoms, promoted an earlier diagnostic age. Among male autistic children or children from families with higher socioeconomic levels, language delay was most significantly associated with an earlier age of diagnosis. Cognitive delay, or adaptive delay, was associated with a later age at diagnosis and presented only in autistic children from families with lower socioeconomic levels. There may be sex or socioeconomic inequality in the diagnostic age for autistic children. More publicity and public education about the diversity of autistic symptoms are urgently needed in the future, especially for low-socioeconomic families

The silicon isotope composition of Ethmodiscus rexlaminated diatom mats from the tropical West Pacific: Implications for silicate cycling during the Last Glacial Maximum

The cause of massive blooms of Ethmodiscus rex laminated diatom mats (LDMs) in the eastern Philippine Sea (EPS) during the Last Glacial Maximum (LGM) remains uncertain. In order to better understand the mechanism of formation of E. rex LDMs from the perspective of dissolved silicon (DSi) utilization, we determined the silicon isotopic composition of single E. rex diatom frustules (δ30SiE. rex) from two sediment cores in the Parece Vela Basin of the EPS. In the study cores, δ30SiE. rex varies from −1.23‰ to −0.83‰ (average −1.04‰), a range that is atypical of marine diatom δ30Si and that corresponds to the lower limit of reported diatom δ30Si values of any age. A binary mixing model (upwelled silicon versus eolian silicon) accounting for silicon isotopic fractionation during DSi uptake by diatoms was constructed. The binary mixing model demonstrates that E. rex dominantly utilized DSi from eolian sources (i.e., Asian dust) with only minor contributions from upwelled seawater sources (i.e., advected from Subantarctic Mode Water, Antarctic Intermediate Water, or North Pacific Intermediate Water). E. rex utilized only ~24% of available DSi, indicating that surface waters of the EPS were eutrophic with respect to silicon during the LGM. Our results suggest that giant diatoms did not always use a buoyancy strategy to obtain nutrients from the deep nutrient pool, thus revising previously proposed models for the formation of E. rex LDMs

Cleaning of marine sediment samples for large diatom stable isotope analysis

International audienceDiatom stable isotope analysis offers considerable potential in palaeoceanography, particularly where carbonate material is scarce or absent. However, extracting pure diatom frustules free of external labile organic matter from marine sediments is an essential requirement for their applications as paleoenvironmental proxies. Here, based largely on previous work, we developed a method including physical separation and chemical oxidation steps to concentrate and clean pure large diatoms from laminated diatom mat and diatomaceous clay sediment samples for their stable isotope analysis. Using the physical separation techniques consisting of the removal of carbonate and excess organic matter, sieving, differential settling, and heavy liquid floatation, pure diatoms can be successfully isolated from the sediment samples with opal concentration more than 10%. Subsequent time oxidation experiment shows that labile organic matter coating pure diatom valves can be effectively removed with 30% H 2 O 2 at 65 ℃ for 2 h. Measurements of δ 13 C after every step of physical separation demonstrate that contaminants and lost diatoms can influence the original diatom stable isotope signal, highlighting the importance of a visual check for dominant diatom size in the initial sample and purity in the final sample. Although the protocol described here was only applied to diatom mats or diatom oozes containing large diatoms (Ethmodiscus rex), we believe that this method can be adapted to common dia-toms of general marine sediment samples. KEY WORDS: large diatom, stable isotope, physical separation, chemical oxidation, Parece Vela basin, palaeoceanography

Western Pacific physical and biological controls on atmospheric CO2 concentration over the last 700 kyr

We present new geochemical evidence of changes in the vertical dissolved inorganic carbon (DIC) distribution in the western tropical Pacific over the last 700 kyr, derived from stable carbon isotope (ä13C) signals recorded in epifaunal benthic (Cibicidoides wuellerstorfi) and thermocline-dwelling planktonic (Pulleniatina obliquiloculata) foraminifera extracted from the Calypso Core MD06-3047. We further analyse the results of a transient numerical experiment of the Last Glacial Maximum (LGM) and the last deglaciation performed with the carbon isotope-enabled earth system model LOVECLIM, to understand the deglacial changes in DIC distribution and verify the proxy-based hypothesis. During glacial periods of the past 700 kyrs, the distinct negative deep water ä13CDIC values obtained from the benthic foraminifera suggest a carbon increase in the deep ocean, which could have been caused by weakening of deep Southern Ocean (SO) ventilation and enhanced marine biological productivity driven by dust-induced iron fertilization. During glacial terminations, a decrease of thermocline ä13CDIC associated with an increase in deep water ä13CDIC indicate a reduced vertical DIC gradient and the net transmission of 12C from the deep waters to the thermocline, caused mainly by the physical process (enhanced SO ventilation). On longer time scales, the largest increase in the Pacific deep carbon reservoir ä13CDIC during the marine isotope stage (MIS) 12/11 transition coincided with the mid-Brunhes climatic shift, which implies that the extent of oceanic carbon release during this interval was much larger than that during other deglaciations since 700 ka B.P. We infer that this could have been caused by reorganization of the oceanic carbon system. These findings provide new insights into the Pleistocene evolution of the carbon-cycle system in the Pacific Ocean

The evolution and control of detrital sediment provenance in the middle and northern Okinawa Trough since the last deglaciation: Evidence from Sr and Nd isotopes

Highlights • Yangtze and Yellow River sediments can be distinguished with SrNd isotope. • Okinawa Trough (OT) sediments derived from the Yangtze River from 18.0 to 10.5 ka. • Yangtze and Yellow River contributed sediment to the OT from 10.5 to 7.0 ka. • Kuroshio Current control Taiwan sediment to the OT after 7.0 ka. • Yellow River sediment was transported to the OT by the coastal current in the last 10.5 ka. Abstract The Okinawa Trough (OT) is a large sink of sediments supplied by the East Asian continent. Identifying the provenance of the OT sediments is key to reconstructing the temporal and spatial variations of the terrigenous supply to this area and is important for understanding the impact of paleoclimatic and paleoceanographic variability on the sediment supply to this marginal sea over the late Quaternary. In this contribution, we show that radiogenic strontium (Sr) and neodymium (Nd) isotopes allow to efficiently distinguish Yellow and Yangtze/Taiwan River detrital sediments, and can be used to reconstruct distinct changes in the provenance of the detrital fraction of marine sediments from the middle and northern OT since the last deglaciation. The Sr and Nd isotope signatures are compared to those of the potential sediment sources, namely the Yellow and Yangtze Rivers, the Taiwan orogen, and volcanic material from the OT and nearby islands, and the relative contributions of these sources are reconstructed. The Sr and Nd isotope compositions of the detrital fraction in the two sediment cores recovered from the middle and northern OT show that the sediments mainly originated from the Yangtze River between 18 and 10.5 ka, which was caused by low sea level and a widely developed channel system on the continental shelf. During the period between 10.5 and 7.0 ka, the rising sea level resulted in elevated Yangtze and Yellow Rivers sediment input into the OT. Simultaneously, large-scale volcanic activity also contributed significant amounts of material to the OT. During the last 7.0 ka, besides important contributions from the Yellow River, the intensification of the Kuroshio Current resulted in increased delivery of sediment from Taiwan to the OT